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In April 1991, the Board of the National Council on 
Measurement in Education (NCME) voted to accept a grant from the 
Kellogg Foundation to: (1) measure teacher competency levels in the 
educational assessment of students; (2) identify a topic for the 
development of a training prototype; and (3) prepare a prototype 
training module on the selected topic. Efforts toward meeting these 
goals are summarized. Goal 1 was achieved through a national survey 
of teacher competencies in the area of educational assessment of 
students responded to by 555 teachers (48 percent response rate) and 
286 administrators (51 percent response). On the basis of that survey 
and other relevant data, the topic "Interpreting and Communicating 
Test Results" was selected for development of a training prototype. 
Efforts toward Goal 3 are in progress. The training modules will 
consist of several components, designed to be delivered in either a 
1-day or 2-day inservice program or through a series of 2-hour 
training sessions. The training modules will consist of a 
context-orienting vignette, course training materials, practice 
exercises, hands-on experiences, small group discussion questions, 
and homework problems. This prototype could serve as o model for 
future development efforts for training teachers in educational 
assessment. (Author/SLD) 
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At the April, 1991 Board of Directors meeting, the NCME Board voted to accept a grant 
from the Kellogg Foundation to (1) measure teacher competency levels in the educational 
assessment of students, (2) identify a topic for development of a training prototype, (3) 
prepare a prototype training module on the selected topic. The purpose of this paper is to 
summarize efforts toward meeting these goals. Specifically, goal one was achieved 
through a national survey of teacher competencies in the area of educational assessment of 
students. On the basis of the re sults of that assessment, and other relevant data, the topic 
"Interpreting and Communicating Test Results" was selected for development of a training 
prototype. Efforts toward goal three are in progress. The training modules will consist of 
several components, designed to be delivered in either a one or two day inservice program 
or through a series of 2 hour training sessions. The training modules will consist of a 
context-orienting vignette, course training materials, practice exercises, hands-on 
experiences, small group discussion questions and homework problems. This prototype 
could serve as a model for future development efforts for training teachers in educational 
assessment. 



Teacher Assessment Literacy: Development of Training Modules 
Report on an NCME-based Kellogg Foundation Grant 

At the April, 1991 Board of Directors meeting, the NCME Board voted to accept a grant 
from the Kellogg Foundation to (1) measure teachers 1 competency level in the assessment 
areas addressed in the Standards for Teacher Competencies in the Educational Assessment 
of Students (NEA, NCME, AFT, 1990); (2) identify a target assessment area based on the 
results of measuring teachers' competencies; and (3) develop a training prototype. Barbara 
S. Plake and James C. Impara serve as co-principal investigators on the project. A 
National Advisory Committee was formed consisting of representatives from the American 
Federation of Teachers (Marsha Berger, recently replaced by Beth Bader), the National 
Education Association (Jeffrey Schneider), and NCME (Tony Nitko and Bill Schafer). 

The purpose of this paper is to summarize efforts toward meeting these goals. 

Goal 1: Measure teacher competency levels in the educational assessment 
of students* 

An instrument was developed to measure the seven competency areas articulated in the 
Standards . The instrument was administered to a national sample of teachers and 
administrators in fall, 1991. The sampling plan involved identification of 6 school districts 
per state, selected to represent urban and rural school district. Through contact with state 
assessment directors, direct contact information was obtained for the district testing 



ERLC 



4 



coordinator (or equivalent) in each sampled school districts. These district testing 
coordinators were then requested to cooperate by randomly sampling 12 teachers from 
across elementary, middle, and secondary levels and 6 administrators from elementary and 
secondary school settings. All total, 98 school systems from 45 states participated. 

Results. A total of 555 (48%) teachers and 286 (51%) administrators returned their test 
materials. The distribution of respondents was widespread across the U.S., representing 
large and small school districts in rural, suburban and urban areas. Overall, teachers 
averaged about 23 out of 35 items correct Highest performance was on the subscale 
measuring teacher knowledge in the areas of administering, scoring and interpreting test 
results; poorest performance was on items measuring the teachers 1 knowledge about 
communicating test results. Teachers who had some measurement training scored 
significantly higher than those who had not; teachers expressing comfort in interpreting 
standardized tests scored significantly higher on the test than did teachers who expressed 
discomfort. 

In addition to items directly measuring the Standards , teachers 1 backgrounds and interests 
in becoming more knowledgeable about assessment information were asked. One question 
of relevance to the training endeavor asked teachers for their preference for the mode of 
training; the clear preference was for inservice delivery of assessment training information. 
More information on the results of this national assessment of teacher competencies in 
educational assessment of students can be found in Plake, Inipara, and Fager (in press). 



Goal 2: Identify a topic for developing a training prototype 



Based on the results of the national survey and other pertinent information, the content area 
of "Interpreting and Communicating test results" was identified as the focus for the 
development of prototype materials. According to the Standards for Teacher Competence 
in the Educational Ass essment of Students , the following conception and applications skills 
are necessary in order for teachers to meet this standard: 

1. Understand and be able to give appropriate explanations of how the 
interpretation of student assessments must be moderated by the student's socio- 
economic, cultural, language, and other background factors. 

2. Be able to explain that assessment results do not imply that such background 
factors limit a student's ultimate educational development 

3. Communicate to students and to their parents or guardians how teachers may 
assess the student's education progress. 

4. Understand and be able to explain the importance of taking measurement errors 
into account when using assessments to make decisions about individual 
students. 

5. Be able to explain the limitations of different informal and formal assessment 
methods. 



6. Be able to explain printed reports of the results of pupil assessments at the 
classroom, school district, state, and national levels* 

Goal 3: Develop a training prototype for the selected topic area 

The training modules will consist of several components, designed to be delivered in either 
a one or two day inservice program or through a series of 2 hour training sessions. The 
training modules will consist of a context-orienting vignette (problem context), instruction 
on the topic, practice exercises, hands-on experiences, small group discussion questions 
and homework problems. This prototype will serve as a model for furore development 
efforts for training teachers in other areas related to assessing students. 

Problem Context A parent-teacher conference was selected as the problem context for the 
"Interpreting and Communicating Test Results" training module. It will be presented in the 
form of a written vignette, instead of other forms of media presentation, such as video or 
audio tape. The reason for this decision was to make the training modules as portable as 
possible; further development of media based problem context presentation could occur 
under another grant effort or by independent developers The vignette will consist of three 
major segments: 

1. Introduction: purposes of the training module described briefly; the focus of the 
vignette is on interpreting and communicating results of student academic 
achievement assessments. 

2. Simulated parent - teacher conference: parent queries a teacher about the 
meaning and interpretation of achievement information, including formal and 
informal assessment results. 



3, Conclusion: key features of the parent - teacher conference identified; 
introduction to the instructional component is provided 



Course Training Material 

The focus of the training material will be on "sensitive" interpretation of test results. The 
course content will emphasize that there are multiple sources of student achievement 
information, involving both informal and formal assessments. Further, it will be noted that 
parent teacher conferences provide the opportunity for parents to be presented with both 
formative and summative evaluations of student achievement Such conferences also 
provide a context for the teacher gather information about possible factors that might 
influence the student's achievements. While the content of the training will focus on 
fundamental precepts of assessment, they will be presented less formal ways. For 
example, the concepts of validity and reliability will be discussed, but not in a formal 
academic sense. Instead, these constructs will be intertwined in the discussion of 
interpreting test results, in particular in the segment on understanding consistencies and 
inconsistencies in achievement information. Also highlighted in an application mode will 
be concepts of the utility of separate assessment component results and possible sources for 
test score variance 

Specific components of the training course content include: 

1. Definitions of different kinds of achievement outcomes 

2. Identification of different purposes of assessments 

3. Determination of summative representation of student performance (e.g., 
methods of assigning grades) 
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4. Specification of relative strengths and weaknesses of students across subject 
areas. 

5. Specification of relative strengths and weaknesses of students as compared to 
other students. 

6. Discussion of relative strengths and weaknesses of students in light of other 
information (i.e., student background, ability, motivation). 

7. Discussion of ways in which parent could be helpful in students school 
achievement 

8. Explanation of achievement in light of inconsistent performance indicators. 
Status of Development of the Training Modules Prototype 

The parent teacher conference vignette is under development The final training module 
prototype will consist of the parent-teacher vignette, the course training instructional 
objectives, a set of practice exercises, hands-on experiences, small group discussion 
questions and developed model responses, and homework problems. 

Future of the Training Module Project 

The Kellogg based NCME grant provided resources for developing only one prototype for 
training teachers in the area of educational assessment of students. It is hoped that other 
opportunities will become available to further the development of more training modules. 
NCME has been communicating with NEA, AFT and some of the Regional Laboratories 
about possible collaboration of continued project efforts. In the meantime, this prototype 
can serve as an illustration of development efforts toward enhancing the assessment literary 
of teachers. 

3 
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